Zeroth-Order Optimization for Composite Problems with Functional Constraints

In many real-world problems, first-order (FO) derivative evaluations are too expensive or even inaccessible. For solving these zeroth-order (ZO) methods that only need function often more efficient than FO sometimes the options. this paper, we propose a novel inexact augmented Lagrangian method (ZO-iALM) to solve black-box optimization which involve composite (i.e., smooth+nonsmooth) objective and functional constraints. Under certain regularity condition (also assumed by several existing works on methods), query complexity of our ZO-iALM is $\tilde{O}(d\varepsilon^{-3})$ find an $\varepsilon$-KKT point for problems with nonconvex constraints, $\tilde{O}(d\varepsilon^{-2.5})$ convex where $d$ variable dimension. This appears be first work develops iALM-based ZO constrained meanwhile achieves results matching best-known up factor $d$. With extensive experimental study, show effectiveness method. The applications span from classical practical machine learning examples such as resource allocation in sensor networks adversarial example generation.